Konstantina Palla , David A . Knowles , Zoubin Ghahramani

This section expands on the analysis of Section 3 in the original manuscript regarding the relation of the SHGP to Hierarchical Gamma (HGP) and Hierarchical Dirichlet processes (HDP) as seen in Table 1. The SHGP is a prior over the weight matrix J as opposed to the HDP which is a prior over the transition matrix P . This difference is crucial, since it allows for direct manipulation of the weights, enabling us to enforce symmetry and thereby make the Markov chain reversible. The SHGP can be viewed as a HGP where symmetry is imposed on the produced weight matrix J . However, there are subtle differences in the construction of the weight matrix. Looking at the Table 1, both processes, the HGP and SHGP, use the Gamma process in a hierarchical way. The HGP constructs each row j in the weight matrix by sampling from the same Gamma process ΓP (α̃, G0),∀j, as opposed to the SHGP where each row is sampled by a Gamma process with a different shape parameter dependent on the corresponding base weight wj . This is a modelling choice and by choosing the base measure μ of the weight matrix to be the G0 rather than the product G0 × G0 the SHGP (with no symmetrization imposed) becomes identical to the HGP.